
<!DOCTYPE html>

<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

    <title>Timeouts &#8212; LAVA 2024.05 documentation</title>
    <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="_static/bootstrap-sphinx.css" />
    <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
    <script src="_static/jquery.js"></script>
    <script src="_static/underscore.js"></script>
    <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
    <script src="_static/doctools.js"></script>
    <script src="_static/sphinx_highlight.js"></script>
    <link rel="shortcut icon" href="_static/favicon.ico"/>
    <link rel="index" title="Index" href="genindex.html" />
    <link rel="search" title="Search" href="search.html" />
    <link rel="next" title="Writing a Lava-Test Test Definition 1.0" href="writing-tests.html" />
    <link rel="prev" title="Writing Tests" href="developing-tests.html" />
    <link rel="canonical" href="https://docs.lavasoftware.org/lava/timeouts.html" />
  
<meta charset='utf-8'>
<meta http-equiv='X-UA-Compatible' content='IE=edge,chrome=1'>
<meta name='viewport' content='width=device-width, initial-scale=1.0, maximum-scale=1'>
<meta name="apple-mobile-web-app-capable" content="yes">
<script type="text/javascript" src="_static/js/jquery-1.12.4.min.js"></script>
<script type="text/javascript" src="_static/js/jquery-fix.js"></script>
<script type="text/javascript" src="_static/bootstrap-3.4.1/js/bootstrap.min.js"></script>
<script type="text/javascript" src="_static/bootstrap-sphinx.js"></script>


  </head><body>

  <div id="navbar" class="navbar navbar-default navbar-fixed-top">
    <div class="container">
      <div class="navbar-header">
        <!-- .btn-navbar is used as the toggle for collapsed navbar content -->
        <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".nav-collapse">
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
        </button>
        <a class="navbar-brand" href="index.html"><span><img src="_static/lava.png"></span>
          LAVA</a>
        <span class="navbar-text navbar-version pull-left"><b>2024.05</b></span>
      </div>

        <div class="collapse navbar-collapse nav-collapse">
          <ul class="nav navbar-nav">
            
                <li><a href="genindex.html">Index</a></li>
                <li><a href="contents.html">Contents</a></li>
            
            
              <li class="dropdown globaltoc-container">
  <a role="button"
     id="dLabelGlobalToc"
     data-toggle="dropdown"
     data-target="#"
     href="index.html">Site <b class="caret"></b></a>
  <ul class="dropdown-menu globaltoc"
      role="menu"
      aria-labelledby="dLabelGlobalToc"><ul class="current">
<li class="toctree-l1"><a class="reference internal" href="index.html">Introduction to LAVA</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="contents.html">Contents</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="glossary.html">Glossary of terms</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="support.html">Getting support</a></li>
</ul>
</ul>
</li>
              
                <li class="dropdown">
  <a role="button"
     id="dLabelLocalToc"
     data-toggle="dropdown"
     data-target="#"
     href="#">Page <b class="caret"></b></a>
  <ul class="dropdown-menu localtoc"
      role="menu"
      aria-labelledby="dLabelLocalToc"><ul>
<li><a class="reference internal" href="#">Timeouts</a><ul>
<li><a class="reference internal" href="#automation-and-timeouts">Automation and Timeouts</a><ul>
<li><a class="reference internal" href="#why-must-timeouts-exist">Why must timeouts exist?</a></li>
<li><a class="reference internal" href="#how-long-should-an-operation-wait">How long should an operation wait?</a></li>
<li><a class="reference internal" href="#operations-involving-third-party-services">Operations involving third party services</a></li>
<li><a class="reference internal" href="#duration-of-actions">Duration of actions</a></li>
<li><a class="reference internal" href="#test-shell-timeouts">Test shell timeouts</a></li>
</ul>
</li>
<li><a class="reference internal" href="#defining-timeouts">Defining timeouts</a></li>
<li><a class="reference internal" href="#job-timeouts">Job timeouts</a><ul>
<li><a class="reference internal" href="#summary-of-the-example-job-timeouts">Summary of the example job timeouts</a></li>
</ul>
</li>
<li><a class="reference internal" href="#default-action-timeouts">Default action timeouts</a></li>
<li><a class="reference internal" href="#default-connection-timeouts">Default connection timeouts</a></li>
<li><a class="reference internal" href="#inheriting-timeouts-from-the-device-configuration">Inheriting timeouts from the device configuration</a></li>
<li><a class="reference internal" href="#individual-action-overrides">Individual action overrides</a></li>
<li><a class="reference internal" href="#individual-connection-overrides">Individual connection overrides</a></li>
<li><a class="reference internal" href="#action-block-overrides">Action block overrides</a></li>
<li><a class="reference internal" href="#skipping-a-test-shell-timeout">Skipping a test shell timeout</a></li>
</ul>
</li>
</ul>
</ul>
</li>
              
            
            
              
                
  <li>
    <a href="developing-tests.html" title="Previous Chapter: Writing Tests"><span class="glyphicon glyphicon-chevron-left visible-sm"></span><span class="hidden-sm hidden-tablet">&laquo; Writing Tests</span>
    </a>
  </li>
  <li>
    <a href="writing-tests.html" title="Next Chapter: Writing a Lava-Test Test Definition 1.0"><span class="glyphicon glyphicon-chevron-right visible-sm"></span><span class="hidden-sm hidden-tablet">Writing a Lav... &raquo;</span>
    </a>
  </li>
              
            
            
            
            
              <li class="hidden-sm"></li>
            
          </ul>

          
            
<form class="navbar-form navbar-right" action="search.html" method="get">
 <div class="form-group">
  <input type="text" name="q" class="form-control" placeholder="Search" />
 </div>
  <input type="hidden" name="check_keywords" value="yes" />
  <input type="hidden" name="area" value="default" />
</form>
          
        </div>
    </div>
  </div>

<div class="container">
  <div class="row">
    <div class="body col-md-12 content" role="main">
      
  <section id="timeouts">
<span id="index-0"></span><span id="id1"></span><h1>Timeouts<a class="headerlink" href="#timeouts" title="Permalink to this heading">¶</a></h1>
<p>V2 provides detailed control over timeouts within testjobs. The job has an
over-arching timeout, each action has a default runtime timeout, each
connection has a default timeout to get a reply and individual actions within
the pipeline can have the action or connection timeout modified in the test
definition or in the device configuration.</p>
<section id="automation-and-timeouts">
<span id="need-for-timeouts"></span><h2>Automation and Timeouts<a class="headerlink" href="#automation-and-timeouts" title="Permalink to this heading">¶</a></h2>
<section id="why-must-timeouts-exist">
<h3>Why must timeouts exist?<a class="headerlink" href="#why-must-timeouts-exist" title="Permalink to this heading">¶</a></h3>
<p>Automation needs to be able to cope with failures during operation and one of
the most common failure modes is that a command never returns because some part
of the setup for that command has not been done, failed during operation or is
not supported for some reason.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>It is not usually the operation which times out which is the reason
for the failure. Typically, a timeout results from one or more <strong>previous</strong>
operations failing. A test job which fails due to a timeout <strong>must always be
investigated</strong> - simply extending the timeout is the wrong approach.
Equally, long timeouts <strong>are directly harmful to other users</strong>, choose your
timeouts carefully and regularly evaluate whether existing timeouts can be
shortened.</p>
</div>
<p>When operating the same test interactively, a human will typically notice that
a step has failed and either not attempt the operation that would later time
out or recognize that the operation should have completed by some point in time
and intervene. It is not always possible to check the success or failure of
operations within a test in an automated fashion - command outputs change from
one distribution to another or from one version of a package to another.</p>
<p>The timeout is a necessary part of automation, it allows test jobs to fail
instead of holding on to the job and the device indefinitely.</p>
</section>
<section id="how-long-should-an-operation-wait">
<span id="calculating-timeouts"></span><span id="index-1"></span><h3>How long should an operation wait?<a class="headerlink" href="#how-long-should-an-operation-wait" title="Permalink to this heading">¶</a></h3>
<p>The best guidance here is to assess how the operation works when you use the
device interactively. Compare over a few runs of the operation and then use a
timeout which is slightly longer than the longest successful operation by
rounding up to the nearest whole minute or hour.</p>
<p>Timeouts do not need to be precise, 2 minutes is better than 90 seconds, but
yet <strong>must not be excessive</strong>. If an operation has routinely taken 5 minutes to
succeed previously and has now suddenly taken 30 minutes, this <strong>needs to be
investigated</strong>. It could easily be a kernel bug, hardware fault, infrastructure
fault or test job error. Increase timeouts <strong>gradually</strong> and keep a sense of
perspective of just what is reasonable to expect an operation to require.</p>
<div class="admonition important">
<p class="admonition-title">Important</p>
<p>Do <strong>not</strong> simply transfer timeouts from V1 jobs! V1 timeouts do
not have the same structure and cannot be easily mapped to individual
actions or operations within an action.</p>
</div>
</section>
<section id="operations-involving-third-party-services">
<h3>Operations involving third party services<a class="headerlink" href="#operations-involving-third-party-services" title="Permalink to this heading">¶</a></h3>
<p>When downloading, uploading or transferring data using third party services it
can be hard to estimate a reasonable timeout. If test jobs start to fail during
such operations, investigate whether the connection to the third party service
can be improved, cached or fixed.</p>
</section>
<section id="duration-of-actions">
<h3>Duration of actions<a class="headerlink" href="#duration-of-actions" title="Permalink to this heading">¶</a></h3>
<p>The duration of every action in a test job is tracked and recorded. This allows
test writers to look at other similar jobs and evaluate the actual duration of
any operation within that testjob. Equally, it allows lab admins to compare
your timeouts against the actual duration of the operation. If your jobs start
to fail and sit idle for long periods waiting for a timeout, you have the
information to hand to fix the timeouts yourself before you get a prompt from
the admins.</p>
</section>
<section id="test-shell-timeouts">
<span id="id2"></span><h3>Test shell timeouts<a class="headerlink" href="#test-shell-timeouts" title="Permalink to this heading">¶</a></h3>
<p>The timeout used by the test action is a single value covering all test
operations. Actual durations are still tracked and recorded, so
excessive timeouts still need to be addressed.</p>
</section>
</section>
<section id="defining-timeouts">
<span id="index-2"></span><span id="id3"></span><h2>Defining timeouts<a class="headerlink" href="#defining-timeouts" title="Permalink to this heading">¶</a></h2>
<p>For the test writer, the timeout is expressed as a single integer value of:</p>
<ul class="simple">
<li><p>seconds,</p></li>
<li><p>minutes,</p></li>
<li><p>hours or</p></li>
<li><p>days</p></li>
</ul>
<p>There is no need to specify sub-divisions or to overflow. Instead of <code class="docutils literal notranslate"><span class="pre">seconds:</span>
<span class="pre">90</span></code> use <code class="docutils literal notranslate"><span class="pre">minutes:</span> <span class="pre">2</span></code> and instead of trying to specify two and a half
minutes, just use <code class="docutils literal notranslate"><span class="pre">minutes:</span> <span class="pre">3</span></code>. Using <code class="docutils literal notranslate"><span class="pre">hours:</span> <span class="pre">2</span></code> when only <code class="docutils literal notranslate"><span class="pre">minutes:</span> <span class="pre">2</span></code>
is required is likely to get you a warning from the admins but using <code class="docutils literal notranslate"><span class="pre">minutes:</span>
<span class="pre">10</span></code> instead of <code class="docutils literal notranslate"><span class="pre">seconds:</span> <span class="pre">600</span></code> is <strong>strongly</strong> recommended.</p>
<p>Although timeouts support <code class="docutils literal notranslate"><span class="pre">days</span></code>, you need to have a <strong>very</strong> good reason to
set such a timeout to avoid being accused of denying access to the device to
other users (including the special <code class="docutils literal notranslate"><span class="pre">lava-health</span></code> user which is used to submit
health checks).</p>
</section>
<section id="job-timeouts">
<span id="job-timeout"></span><h2>Job timeouts<a class="headerlink" href="#job-timeouts" title="Permalink to this heading">¶</a></h2>
<p>The entire test job has a single over-arching timeout. This means that no
matter how long any action or connection timeout is set within the test job, if
the test job duration increases above the <strong>job timeout</strong> then the <em>slave</em> will
terminate the job and set the status as <strong>Incomplete</strong>.</p>
<p>The first reason for this timeout is so that individual actions or connections
can have freedom to set timeouts but the testjob still fails if more than one
or two of the operations take significantly longer than anticipated.</p>
<p>The second reason for a job timeout is that it allows the UI to derive an
estimate of how long the job will take to inform other users who may be waiting
for their jobs to start on the busy devices.</p>
<pre class="code yaml literal-block"><code><span class="name tag">timeouts</span><span class="punctuation">:</span><span class="whitespace">
  </span><span class="name tag">job</span><span class="punctuation">:</span><span class="whitespace">
    </span><span class="name tag">minutes</span><span class="punctuation">:</span><span class="whitespace"> </span><span class="literal scalar plain">15</span><span class="whitespace">
  </span><span class="name tag">action</span><span class="punctuation">:</span><span class="whitespace">
    </span><span class="name tag">minutes</span><span class="punctuation">:</span><span class="whitespace"> </span><span class="literal scalar plain">5</span><span class="whitespace">
  </span><span class="name tag">connection</span><span class="punctuation">:</span><span class="whitespace">
    </span><span class="name tag">minutes</span><span class="punctuation">:</span><span class="whitespace"> </span><span class="literal scalar plain">4</span><span class="whitespace">
  </span><span class="name tag">actions</span><span class="punctuation">:</span><span class="whitespace">
    </span><span class="name tag">u-boot-commands</span><span class="punctuation">:</span><span class="whitespace">
      </span><span class="name tag">minutes</span><span class="punctuation">:</span><span class="whitespace"> </span><span class="literal scalar plain">3</span><span class="whitespace">
  </span><span class="name tag">connections</span><span class="punctuation">:</span><span class="whitespace">
    </span><span class="name tag">lava-test-shell</span><span class="punctuation">:</span><span class="whitespace">
      </span><span class="name tag">minutes</span><span class="punctuation">:</span><span class="whitespace"> </span><span class="literal scalar plain">4</span></code></pre>
<p>The <code class="docutils literal notranslate"><span class="pre">timeouts</span></code> block specifies the job timeout, as well as the
<a class="reference internal" href="#default-action-timeouts"><span class="std std-ref">Default action timeouts</span></a> (5 minutes in this example) and
<a class="reference internal" href="#default-connection-timeouts"><span class="std std-ref">Default connection timeouts</span></a> (4 minutes in this example).</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="#action-block-timeout-overrides"><span class="std std-ref">Action block overrides</span></a>,
<a class="reference internal" href="#individual-action-timeout-overides"><span class="std std-ref">Individual action overrides</span></a> and
<a class="reference internal" href="#individual-connection-timeout-overrides"><span class="std std-ref">Individual connection overrides</span></a>.</p>
</div>
<section id="summary-of-the-example-job-timeouts">
<h3>Summary of the example job timeouts<a class="headerlink" href="#summary-of-the-example-job-timeouts" title="Permalink to this heading">¶</a></h3>
<ul>
<li><p>The test job will not take longer than <strong>15 minutes</strong> or it will timeout.
This will happen irrespective of which action is currently running or how
much time that action has before it would timeout.</p></li>
<li><p>No one action (deploy, boot or test) will take longer than <strong>5 minutes</strong> or
that action will timeout. Each operation within the action (the action class)
will pass on the remaining time to the next operation. Enable the debug logs
at the top of the log page to see this as a decreasing <code class="docutils literal notranslate"><span class="pre">timeout</span></code> value with
each <code class="docutils literal notranslate"><span class="pre">start</span></code> operation:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>start: 1.3.4 compress-overlay (timeout 00:04:06)
end: 1.3.4 compress-overlay (duration 00:00:03)
start: 1.3.5 persistent-nfs-overlay (timeout 00:04:03)
</pre></div>
</div>
</li>
<li><p>No one connection will take longer than <strong>2 minutes</strong> or the action will
timeout. Connection timeouts are between prompts, so this is the maximum
amount of time that any operation within the action can take before the
action determines that there is not going to be any more output and to fail
as a timeout. <strong>Actions typically include multiple connections</strong>, each with
the same timeout. Connection timeouts are not affected by previous
connections, each time a command is sent, the action expects to find the
prompt again within the same connection timeout.</p></li>
<li><p>All timeouts in this top level section can be overridden later in the test
job definition.</p></li>
</ul>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="#test-shell-timeouts"><span class="std std-ref">Test shell timeouts</span></a></p>
</div>
</section>
</section>
<section id="default-action-timeouts">
<span id="id4"></span><h2>Default action timeouts<a class="headerlink" href="#default-action-timeouts" title="Permalink to this heading">¶</a></h2>
<p>An action timeout covers the entire operation of all operations performed by
that action. Check the V2 logs for lines like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">start</span><span class="p">:</span> <span class="mf">1.1.1</span> <span class="n">http_download</span> <span class="p">(</span><span class="n">timeout</span> <span class="mi">00</span><span class="p">:</span><span class="mi">05</span><span class="p">:</span><span class="mi">00</span><span class="p">)</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">end</span><span class="p">:</span> <span class="mf">1.1.1</span> <span class="n">file_download</span> <span class="p">(</span><span class="n">duration</span> <span class="mi">00</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">25</span><span class="p">)</span>
</pre></div>
</div>
<p>The action timeout <code class="docutils literal notranslate"><span class="pre">00:05:00</span></code> comes from this part of the job definition:</p>
<pre class="code yaml literal-block"><code><span class="name tag">timeouts</span><span class="punctuation">:</span><span class="whitespace">
  </span><span class="name tag">job</span><span class="punctuation">:</span><span class="whitespace">
    </span><span class="name tag">minutes</span><span class="punctuation">:</span><span class="whitespace"> </span><span class="literal scalar plain">15</span><span class="whitespace">
  </span><span class="name tag">action</span><span class="punctuation">:</span><span class="whitespace">
    </span><span class="name tag">minutes</span><span class="punctuation">:</span><span class="whitespace"> </span><span class="literal scalar plain">5</span></code></pre>
<p>The complete list of actions for any test job is available from the job
definition page, on the pipeline tab.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Not all actions in any one pipeline will perform any operations.
Action classes are idempotent and can skip operations depending on the
parameters of the testjob. Hence some actions will show a duration of
<code class="docutils literal notranslate"><span class="pre">00:00:00</span></code>.</p>
</div>
</section>
<section id="default-connection-timeouts">
<span id="id5"></span><h2>Default connection timeouts<a class="headerlink" href="#default-connection-timeouts" title="Permalink to this heading">¶</a></h2>
<p>A connection timeout covers each single operation of sending a command to a
device and getting a response back from that device. A new connection timeout
is used for each operation of sending a command to the device. For example,
when sending a list of commands to a bootloader, each complete line has the
same connection timeout which is reset back to zero for the subsequent line.</p>
<p>Connection timeouts can be much shorter than action timeouts, especially if the
action needs to send multiple lines of commands.</p>
</section>
<section id="inheriting-timeouts-from-the-device-configuration">
<span id="device-configuration-timeouts"></span><h2>Inheriting timeouts from the device configuration<a class="headerlink" href="#inheriting-timeouts-from-the-device-configuration" title="Permalink to this heading">¶</a></h2>
<p>In addition, individual device types can set an action override or connection
override for all pipelines using devices of that type. This is to allow for
certain devices which need to initialize certain hardware that takes longer
than most other devices with similar support.</p>
<p>Details of these timeouts can be seen on the device type page on the <em>Support</em>
tab and can be overridden using the overrides in the test job.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The actual timeout for each action is computed by taking the device
configuration and overriding the values with the timeouts from the job
definition.
The timeout will be the first defined value in:
<a class="reference internal" href="#action-block-timeout-overrides"><span class="std std-ref">Action block overrides</span></a>,
<a class="reference internal" href="#individual-action-timeout-overides"><span class="std std-ref">Individual action overrides</span></a> and <a class="reference internal" href="#default-action-timeouts"><span class="std std-ref">Default action timeouts</span></a>.</p>
</div>
</section>
<section id="individual-action-overrides">
<span id="individual-action-timeout-overides"></span><h2>Individual action overrides<a class="headerlink" href="#individual-action-overrides" title="Permalink to this heading">¶</a></h2>
<p>For fine-grained control over action timeouts, individual actions can be named
in the timeout block at the top of the test job submission and assigned a
specific timeout which can be longer or shorter than the default or the action
block override.</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">timeouts</span><span class="p">:</span>
<span class="w">  </span><span class="nt">actions</span><span class="p">:</span>
<span class="w">    </span><span class="nt">http-download</span><span class="p">:</span>
<span class="w">      </span><span class="nt">minutes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">2</span>
</pre></div>
</div>
</section>
<section id="individual-connection-overrides">
<span id="individual-connection-timeout-overrides"></span><h2>Individual connection overrides<a class="headerlink" href="#individual-connection-overrides" title="Permalink to this heading">¶</a></h2>
<p>For fine-grained control over connection timeouts, individual actions can be
named in the timeout block at the top of the test job submission and assigned a
specific connection timeout which can be longer or shorter than the default.</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">timeouts</span><span class="p">:</span>
<span class="w">  </span><span class="nt">connections</span><span class="p">:</span>
<span class="w">    </span><span class="nt">http-download</span><span class="p">:</span>
<span class="w">      </span><span class="nt">minutes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">2</span>
</pre></div>
</div>
</section>
<section id="action-block-overrides">
<span id="action-block-timeout-overrides"></span><h2>Action block overrides<a class="headerlink" href="#action-block-overrides" title="Permalink to this heading">¶</a></h2>
<p>The test job submission action blocks, (<code class="docutils literal notranslate"><span class="pre">deploy</span></code>, <code class="docutils literal notranslate"><span class="pre">boot</span></code> and <code class="docutils literal notranslate"><span class="pre">test</span></code>) can
also have timeouts. These will override the default action timeout for all
actions within that block. Action blocks are identified by the start of the
<a class="reference internal" href="glossary.html#term-action-level"><span class="xref std std-term">action level</span></a> and the timeout value is set within that action block:</p>
<pre class="code yaml literal-block"><code><span class="name tag">actions</span><span class="punctuation">:</span><span class="whitespace">
</span><span class="punctuation indicator">-</span><span class="whitespace"> </span><span class="name tag">deploy</span><span class="punctuation">:</span><span class="whitespace">
    </span><span class="name tag">timeout</span><span class="punctuation">:</span><span class="whitespace">
      </span><span class="name tag">minutes</span><span class="punctuation">:</span><span class="whitespace"> </span><span class="literal scalar plain">3</span></code></pre>
<p>The default timeout for each action within this block will be set to the specified
value.</p>
<p>The timeout for individual actions in a block can also be redefined in the
timeouts section within the block:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">actions</span><span class="p">:</span>
<span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">deploy</span><span class="p">:</span>
<span class="w">    </span><span class="nt">timeout</span><span class="p">:</span>
<span class="w">      </span><span class="nt">minutes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">5</span>
<span class="w">    </span><span class="nt">timeouts</span><span class="p">:</span>
<span class="w">      </span><span class="nt">http-download</span><span class="p">:</span>
<span class="w">        </span><span class="nt">minutes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">1</span>
</pre></div>
</div>
<span class="target" id="individual-action-block-timeout-overrides"></span></section>
<section id="skipping-a-test-shell-timeout">
<span id="index-3"></span><h2>Skipping a test shell timeout<a class="headerlink" href="#skipping-a-test-shell-timeout" title="Permalink to this heading">¶</a></h2>
<p>In some cases, a test shell action is known to hang or otherwise cause
a timeout.</p>
<p>If the device is capable of booting twice in a single test job
(<em>deploy</em>, <em>boot</em>, <em>test</em>, <em>boot</em>, <em>test</em>) then the first test action
timeout can set <code class="docutils literal notranslate"><span class="pre">skip:</span> <span class="pre">true</span></code> which stops the job finishing as
Incomplete if the timeout occurs. The job will then continue to the
second boot action, allowing the device to reset to a known state and
start the second test action.</p>
<p>If the first test action does not timeout, the job completes that
test action and executes the second boot action as normal.</p>
<p>Test writers should consider using utilities like the <code class="docutils literal notranslate"><span class="pre">timeout</span></code>
command inside their own test shell scripts to retain control within
the currently executing test shell scripts. Skipping a test shell
timeout is intended for tests which may cause a kernel panic or other
deadlock of the currently executing test shell.</p>
<div class="admonition important">
<p class="admonition-title">Important</p>
<p>There are limitations to what can be achieved here:</p>
<ul class="simple">
<li><p>Lava Test Shell is <strong>not re-entrant</strong> - it is not
possible to restart or return to the previously executing test
shell. The second test action is a separate test shell and the
boot action <strong>must</strong> be defined by the test writer at submission.</p></li>
<li><p>Timeouts in <a class="reference internal" href="glossary.html#term-MultiNode"><span class="xref std std-term">MultiNode</span></a> test jobs <strong>cannot be skipped</strong>.</p></li>
<li><p>The timeout itself must still occur - the test job must wait,
after the error has occurred, until the timeout. Ensure that the
timeout value is still long enough to cover the actual execution
time if the test shell action did not hang or fail.</p></li>
<li><p>This support will <strong>not</strong> protect the DUT in the case of a
destructive test shell failure. If the test shell action simply
takes too long because, for example, a parameter has been missed
and the script is deleting all of <code class="docutils literal notranslate"><span class="pre">/usr/lib/</span></code> instead of
<code class="docutils literal notranslate"><span class="pre">/home/test/usr/lib/</span></code>, the DUT will likely fail to reboot
without a new deployment regardless of the skip support.</p></li>
<li><p>This support is intended for predictable test shell errors.
Support must be planned into the test job before submission.
Design your test shell definition and test result handling
carefully. Any expected test case results from the test shell
definition which might have occurred after the timeout will be
completely missing. It is recommended to put the operation which
is expected to fail as the last command in the test shell
definition before the test action would normally end.</p></li>
<li><p>Timeouts are immediate, aggressive and external to the test shell.
There is no opportunity for the active test shell to respond or
handle the timeout. The currently executing process will disappear
and filesystems will <strong>not</strong> be unmounted - power is simply
removed from the <a class="reference internal" href="glossary.html#term-DUT"><span class="xref std std-term">DUT</span></a>. This could affect the ability of the
DUT to execute the next test shell after a reboot, for example if
the filesystem cannot be mounted or the previous test action
failed in the middle of an operation which relies on filesystem
locks.</p></li>
</ul>
<p>Test writers are wholly responsible for cleaning up any artefacts of
the failed test shell at the start of the second test shell. For
example:</p>
<ul>
<li><p>if the first test shell fails when making persistent changes to
the filesystem(s) on the DUT, filesystem corruption is possible
which could cause the second boot action to fail.</p></li>
<li><p>Test actions which install packages are likely to leave stale lock
files in place, incompletely installed packages and other breakage.</p>
<p>Use <a class="reference internal" href="writing-tests.html#test-definition-portability"><span class="std std-ref">portable test shell definitions</span></a> and ensure the integrity of the
packages on the DUT before trying to make more persistent changes
in the second test action block. Assume that the second test
action will start in a broken system if no deploy action is
specified before the second boot action.</p>
</li>
</ul>
</div>
<p>For some devices a <em>deploy</em> action is also needed to get to a point
where the device will boot successfully. (This is also a way of
ensuring that filesystem corruption issues are avoided - by
re-deploying the filesystem itself in a known clean state.)</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">test</span><span class="p">:</span>
<span class="w">   </span><span class="nt">timeout</span><span class="p">:</span>
<span class="w">     </span><span class="nt">minutes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">5</span>
<span class="w">     </span><span class="nt">skip</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
<span class="w">   </span><span class="nt">definitions</span><span class="p">:</span>

<span class="c1"># ... rest of the first test action block</span>

<span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">boot</span><span class="p">:</span>
<span class="w">   </span><span class="nt">timeout</span><span class="p">:</span>
<span class="w">     </span><span class="nt">minutes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">2</span>

<span class="c1"># ... rest of the second boot action block</span>

<span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">test</span><span class="p">:</span>
<span class="w">   </span><span class="nt">timeout</span><span class="p">:</span>
<span class="w">     </span><span class="nt">minutes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">5</span>
<span class="w">   </span><span class="nt">definitions</span><span class="p">:</span>

<span class="c1"># ... rest of the second test action block</span>
</pre></div>
</div>
</section>
</section>


    </div>
      
  </div>
</div>
<footer class="footer">
  <div class="container">
    <p class="pull-right">
      <a href="#">Back to top</a>
      
    </p>
    <p>
        &copy; Copyright 2010-2019, Linaro Limited.<br/>
      Created using <a href="http://sphinx-doc.org/">Sphinx</a> 5.3.0.<br/>
    </p>
  </div>
</footer>
  </body>
</html>